How to create text files in Hadoop?

Member

by cyril , in category: Other , a year ago

How to create text files in Hadoop?

Facebook Twitter LinkedIn Telegram Whatsapp

1 answer

by bernard.rippin , a year ago

@cyril 

To create a text file in Hadoop, you can use the Hadoop File System (HDFS) command hdfs dfs -put to copy a local file to the Hadoop file system. Here is an example:

1
hdfs dfs -put myfile.txt /user/hadoop/


This will create a text file myfile.txt in the HDFS directory /user/hadoop/.


Alternatively, you can also use the Hadoop Streaming utility to create a text file in HDFS. This utility allows you to create and manipulate files in HDFS using any programming language that can read from standard input and write to standard output.


For example, you can use the echo command to create a text file in HDFS like this:

1
echo "Hello, World!" | hadoop fs -put - /user/hadoop/hello.txt


This will create a text file hello.txt in the HDFS directory /user/hadoop/ containing the text "Hello, World!".


You can also use the Hadoop FS command hadoop fs -touchz to create an empty text file in HDFS. For example:

1
hadoop fs -touchz /user/hadoop/empty.txt


This will create an empty text file empty.txt in the HDFS directory /user/hadoop/.